What is custom sharding, and how does it change the way multitenant data is distributed across a cluster?

Custom sharding lets you choose the key that determines which shard a point goes to, instead of letting Qdrant hash the point ID. You declare shard keys (typically tenant IDs, regions, or index names) when you create or extend the collection, and each point carries a shard key that tells the cluster which shard it belongs to. The practical effect is that all points for a given tenant land on the same shard, and a query scoped to that tenant can be routed to that single shard rather than fanning out to every shard in the cluster. In a multitenant system, this is the difference between a query touching one shard and a query touching all of them, which is a large latency and cost difference at scale.

The mechanism relies on two things: the shard key must be known at write time (it is part of the upsert request), and the query must specify the same shard key to benefit from routing. If either is missing, the operation falls back to fan-out. Custom sharding also gives you control over data placement, which matters for isolation: you can put a large tenant on its own shard to prevent it from affecting others, or you can group small tenants onto a shared shard to avoid creating thousands of tiny shards. The trade-off is that the distribution is now your responsibility. If tenants are unevenly sized, some shards will be much larger than others, and a single hot tenant can dominate a node. You also lose the automatic even distribution that hash sharding provides. In practice, the right shard key is medium-cardinality: tenant ID or region works well; user ID or document ID does not, because it creates far too many shards. Cross-tenant queries - rare in most multitenant systems but real in some - become expensive because they have to fan out to all shards.

The trade-off is query routing and isolation against even distribution and operational simplicity. Custom sharding is clearly the right choice for multitenant systems where most queries are tenant-scoped, which is the common case. It is the wrong choice when queries are inherently global - for example, a single search index that everyone queries without a tenant filter - because then every query fans out anyway and you have given up the even distribution for nothing. The common mistake is choosing a shard key with too high a cardinality, like user ID, which creates a shard per user and destroys the benefit while creating enormous operational overhead. The second mistake is forgetting that a query without the shard key still fans out - the routing benefit only exists if the client actually specifies the key. The third mistake is not planning for tenant growth: a shard key that starts as a small tenant can grow into a hot shard, and you need a strategy (dedicated shard, rebalancing) before that happens. Version note: sharding_method and the shard key API are relatively recent additions and have changed across releases - the exact way to create a shard key, the supported key types, and the query syntax for selecting a shard have all evolved.

Version-dependent: the sharding_method enum, create_shard_key, and the shard_key_selector parameter on upsert and query are part of a relatively recent Qdrant API. Earlier versions supported only hash-based sharding, or exposed custom routing through a different mechanism. The supported shard key types (keyword versus integer) and the way a query selects a shard have also changed. If you are designing a multitenant system around custom sharding, verify the API on your version and plan for the fact that adding a shard key is not the same as rebalancing existing data - existing points keep their placement, so a key change usually means a migration.